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ABSTRACT 

In 1990, the Online Catalog Advisory Committee (OCAC) 
of the University of Illinois at Urbana-Champaign (UIUC) charged a 
subcommittee with developing and implementing ways to measure the 
impact of the expanded online catalog on staff, users, collections, 
and library operations. The UIUC's online public access catalog, 
ILL1NET (Illinois Library and Information Network) Online, links the 
circulation systems of about AO institutions in Illinois. The 
subcommittee reviewed literature in this area and developed a 
two-stage study, with baseline and comparative data at one year* 
Instruments to measure impact were developed and introduced to staff 
in orientations at 36 sites. Over 8,000 questions were asked of 
university patrons and external users of the library in each of the 
two years. Study results make it overwhelmingly clear that the new 
online services are very popular and are being used at an increasing 
rate. It is important that staff and users be aware of how to make 
effective use of these resources. Recommendations are made for 
improved training for staff and users, improved and enhanced data 
collection for evaluation, improved resource availability and 
enhanced hardware, and increased attention to the availability of 
periodicals. Appendixes contain the data collection instruments, a 
sample of the data collection code, and 11 statistical tables. 
(Contains 20 references.) (SLD) 
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EXECUTIVE SUMMARY: Final Report of the OCAC Measurement & Evaluation Subcommittee (6/22/93) 

In 1990, the UIUC Library's Online Catalog Advisory Committee charged a new subcommittee, the 
Measurement and Evaluation Subcommittee with measuring the effect of this expanded access on the Library. 
The specific charge to the Subcommittee was to "develop and implement ways to measure the impact of the 
expanded Online Catalog on staff, users, collections and library operations; and to evaluate the expanded 
Online Catalog with respect to CD-ROM databases, searching periodical databases online through dial-access 
and print indices." The Subcommittee's work spanned a period of two years, with its charge being divided 
into a series of measurement and analysis activities: 

* Reviewed the research literature in the area; 

* Developed a two-stage study: a baseline stage, measured just prior to the implementation of the 
expanded access; and a comparative stage, to be measured one year later; 

* Developed a series of hypotheses which would form the basis of the data collection and analyses; 

* Formed teams to devise and test the forms needed to collect the data; 

* Provided staff orientation at 36 sites to promote consistency in data collection; 

* Input the data in machine readable form; and converted it into a form compatible with the SPSS 
statistical analysis program; 

* Analyzed the data to test the hypotheses and held the analyses to very strict levels of statistical 
significance. 

CONCLUSIONS 

UIUC patrons and external users of the Library collections are shifting from what could be termed 
'traditional printed library resources' to electronic resources. Although not all users have moved toward 
electronic resources in the same proportion, all have shifted in that direction. All that distinguishes one group 
of users from another is the magnitude of the shift. 

There was virtually no change in the absolute number of questions asked about serials and periodical 
finding tools from 1991 to 1992, dropping from 8258 to 8184. However, there were substantial changes in the 
types of questions asked. There was an 18% drop in questions relating to serials holdings (significant to the 
.101 level) and a 12% drop in questions about serial locations (.176), totaling 903 questions 'not asked' in 
these two areas. In 1992, there were 1072 questions asked about how to use and interpret the Expanded 
Online Catalog. Between 1991 and 1992 patrons' overall use of electronic resources to find journal articles 
rose from an average of 25% to an average of 52%. Further, it was revealed that frequent and moderate users 
of the Online Catalog made a widespread shift to the use of electronic resources for finding journal articles, 
while patrons who rarely used the online catalog tended use self-supplied sources to find journal articles in 
the Library. 

RECOMMENDATIONS 

The point that became overwhelmingly clear during this study was the popularity of the new online services, 
and the need for staff and users to be informed about how to make effective use of these resources. Based 
on the results of the data analysis, the Subcommittee makes the following recommendations for the Library 
to consider: 

1. Develop and implement a comprehensive training program to insure consistency of staff and user 
expertise with electronic resources throughout the Library organization; 

2. Devise and implement ongoing methods to collect, examine, and analyze data that reflect the effect 
of electronic access to journal article information on staff, collection, and users; 

3. Investigate the full range of resources that might be added to the Online Catalog to serve its diverse 
users, both local and external; 

4. Add terminals to provide greater on-site access to our expanding online resources; 

5. Evaluate the effect of the unavailability of journal articles at the UIUC Library due to theft, 
mutilation, or journal cancellations on access to periodical literature by Library patrons. 
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I. Introduction 

A, Statement o f the Problem 

Many research libraries with online public access catalogs have expanded this search 
environment to include journal citation and abstract information. The University of Illinois 
at Urbana-Champaign Library identified the inclusion of electronic access to journal article 
information as an important priority, based on the assumption that it would improve patrons 9 
access to periodical literature. The determination of appropriate levels of staff service and 
the resources necessaiy in order for users to realize the maximum benefits of expanded 
access have also been of primary concern. This paper describes the methods employed to 
analyze information on periodical use and its associated library support activities, both 
before and after the expansion of the online catalog to include journal citation databases. 

B. Background 

1. Description of ILLINET Online 
The University of Illinois' online public access catalog, ILLINET Online, was developed 
in the late 1970's and early 1980's. A shared system (ILLINET Online) now links the 
circulation systems of approximately 40 institutions in Illinois. Additionally, dial and Internet 
access is available to an unlimited range of users. A user interface that promotes the 
concept of the scholar's workstation (Mischo, et. al., 1990) has been made available within 
the UIU£ Library, and it continues to evolve as new technical applications and services are 
integrated into the catalog. 

2. Mounting of Journal Article databases on ILLINET Online 
In June, 1991, the Library began to provide electronic access to a number of databases 
of journal article, indexes, including Current Contents, several Wilson indexes, ERIC, and 
CARL Uncover. These databases were made available from the same terminals that 
provided access to the Library's Online catalog, ILLINET Online. A locally-developed 
interface facilitated natural language searching of all of these databases, with the exception 
of CARL Uncover, which provides its own interface. Since access to journal citation 
databases had now become an added function in the existing online catalog, the library 



named the sum total of these services ILLINET Online Plus (IO+). While it is apparent 
that these developments have had an effect on the level of use of the Library's collections 
and services, the degree of effect on the Library's collections, staff, and users was speculative 
at the outset of this project. 

C. Charge to the Subcommittee 

The Library's Online Catalog Advisory Committee (OCAC) advises the University 
Librarian on matters concerning the OP AC, and coordinates the activities of technical and 
user support groups, as well as the Library's professional and support staff to implement 
needed changes. In the spring of 1990, the Measurement and Evaluation Subcommittee of 
the OCAC was formed with the charge to "develop and implement ways to measure the 
impact of the expanded Online Catalog on staff, users, collections, and library operations; 
and to evaluate the expanded Online Catalog with respect to CD-ROM databases, searching 
periodical databases online through dial-access, and print indices." This report describes 
the research design and methods, presents the findings of a two-year study, and makes 
recommendations for future development in the area of enhanced periodical access in the 
UIUC Library. 
II. Literature Review 

Recognizing that a number of institutions had already expanded their online catalogs to 
include journal article access, a thorough exploration of the research literature was 
undertaken and the work of other researchers considered. The existing literature provided 
either detailed accounts of projects at other institutions, or concentrated on assessing system 
retrieval performance. Although not all works are cited in this report, a bibliography of 
resources consulted by the Subcommittee is included. 

The current literature contributed to the group's ideas about what the expanded search 
environment would include, what changes in iser behavior could be anticipated, what effect 
on library services, staff, and activities would be the consequence of these user behavior 
changes, and how this effect had been measured in other studies. The Melvyl Medline 
Project ("Final Report," 1990) was of particular interest, as it described a combination of 
user transaction log data and user survey data to evaluate the provision of expanded access. 
The assessment study employed at the National Library of Medicine employed verification 
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and limits testing to rate the technical performance of a new automated system, and a 
multiple-technique approach to assess user acceptance of the system (Siegel, etah, 1984). 
However, it was determined that these evaluative techniques focused on system 
performance, and thus, were not applicable to our study. 

The difficult fiscal constraints currently affecting all libraries, manifested in acquisition 
budget reductions and staffing cutbacks, further convinced the group that it was necessary 
to design a study that introduced measures of the effect of this new service on library users, 
collections and operations. 
III. Methodology 

A, Research Design 

How does expanded access to the periodical literature affect the library? This general 
question framed the research design used in this study. We were presented with an 
opportunity to carry out an experiment, albeit without many of the controls on the 
experimental environment that we would have preferred under laboratory conditions. The 
decision to expand access to the periodical literature through our online catalog was already 
made. Fortunately, the time required to implement the expanded access capability was long 
enough to enable the investigators to collect baseline data before the expanded services were 
made available to the user community. This baseline data was used, in comparison with 
similar information collected after implementation, to determine what effects the expanded 
access had on user journal article-seeking activities, staff, collections, and equipment. 
1. Definitions 

The terms 'periodical* and 'journal* are used throughout this report to refer to the 
specific type of serial publications most often indexed by print and electronic indexing and 
abstracting services. These terms are employed when we refer to the specific information 
patrons seek (e.g., a journal article). The term 'serial' is used as a broader reference to 
serial titles. 

In the context of this study, 'access' was defined as the use of periodical literature per 
user. There are two types of access: physical (the actual use of an item); and intellectual 
(the opportunity to use an item). Therefore, given the same number of users, accessibility 
can be said to increase if more periodical literature is used (either physically or 
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intellectually) or examined than during a previous, comparable period in time. It was 
posited that physical and intellectual access (measured by the ratio of the number of 
periodicals used per user) to the periodical literature may be increased or decreased due to 
the existence of certain conditions. 

It was posited that increased access could affect several elements of staff costs and 
workload, such as an increase in the number of reference questions concerning periodicals, 
an increase in re-shelving, circulation, and interlibrary loan requests for periodical articles, 
more journal theft and mutilation, and more wear-and-tear on support equipment such as 
printers, microform reader/printers, and photocopy machines. 

2. Measurement Method 

For the baseline measurements, data was collected during two weeks in February, 1991. 
This took place before the implementation of the expanded access capabilities, which 
occurred in May of 1991. Comparable data was then collected during two weeks of 
February, 1992, almost one year after these access tools had been made available to the user 
community. Because of the vital role the Library staff played in data collection, we 
conducted an extensive orientation program that focused on the goals of the study and the 
importance of staff cooperation on its success. We identified the data sampling time to 
coincide with the highest level of use of the Library compatible with the effort of data 
collection; thus we were constrained from doing the study during the absolute peak period 
of use of the Library. Each set of data was analyzed separately and then compared (see 
Limitations ^. These data were collected on a library-wide basis at 36 data gathering sites. 
They consisted of the following measurements, all dealing with serials: 

1. Re-shelving of bound serial volumes and unbound issues; 

2. Photocopy machine and computer printer servicing; 

3. Serial mutilation; 

4. Citation sources used by patrons to find journal articles; 

5. Interlibrary loan requests for photocopies of journal articles; 

6. Reference questions asked about serials; 

7. Demographic information on the user population. 

3. Hypotheses 

Several hypotheses were developed in conjunction with the study, all of which were cast 
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in the form of null hypotheses: 

HI. Expanded access would have no significant effect on the number of bound 
serial volumes and unbound issues reshelved. 

H2. Expanded access would not significantly increase the amount of photocopy 
machine and computer printer servicing. 

H3. Expanded access would not significantly increase the amount of serial 
mutilation. 

H4. Expanded access would have no significant effect on the types of citation 
sources chosen by patroas to identify journal articles. 

H5. Expanded access would not significantly increase the number of interlibrary 
loan requests for photocopies of journal articles. 

H6. Expanded access would not significantly increase the number of reference 
questions asked about serials. 

H7. Expanded access would not cause a significant change in the type of reference 
questions asked about serials. 

H8. A user's status in the University community is not a predictor of journal article 
seeking behavior. 

H9. The frequency with which a patron uses the online catalog is not a predictor 
of use of IO+. 

H10. Expanded access would not cause a significant change in the journal article 
seeking activities of users in libraries clustered into broad subject disciplines 
(Arts and Humanities, Life Sciences, Physical Sciences, Social Sciences, 
General population (Reference, Documents, Newspaper, Undergraduate, 
University High School, Circulation and Bookstacks). 
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4. Assumptions 

There were several assumptions made about the environment in which this study was 
conducted during the periods when both the baseline and comparative data were gathered: 

1. User Community; It was assumed that the user community of the University of 
Illinois at Urbana-Champaign remained stable in terms of numbers of faculty, 
students, and other users from 1991 to 1992. Information from the campus indicates 
no significant changes in any group of users during this time. 

2. Number of serials physically available: It was assumed that the number of serial titles 
physically available in the Library remained approximately the same. 

3. Ease of borrowing: It was assumed that the ease of physically borrowing an item 
either on-site or through interlibrary loan did not change. 

4. Reshelving instructions: It was assumed that instructions to users regarding 
reshelving serials did not change. 

5. Equipment: It was assumed that the quality and quantity of photocopiers and 
computer printers did not change. 

6. User behavior: It was assumed that there was nothing to change user behavior with 
regard to mutilation of serials, 

7. ILL: It was assumed that there was no change in interlibrary loan procedures that 
would affect the number of requests made. 

8. Reference: It was assumed that there was no change made in reference policies to 
account for either increased or decreased use. 

9. Library hours: It was assumed that there was no susbtantial change in library hours, 
thereby affecting the amount of time available for serial access. 

10. Serial availability: There were several assumptions relating to increased access to 
serial publications. It was assumed that: 

a) There was an increase in sources available for locating journal articles in the 
online catalog; 

b) There was an increase in the amount of location and holdings information 
available for serials; 



c) The online catalog interface made it convenient to find location information 
about serials; 

d) There was an increase or change in CD-ROM databases. 

1L Representativeness of samples: It was assumed that the two samples taken were 
representative of the population being studied. 

5. Procedures 

Data gathering forms, corresponding to the areas listed above, were designed and refined 
by the committee, and by pilot testing among staff and users (Appendix A.) Training 
sessions were held for those who would be gathering the data. This was done in order to 
make data gathering as uniform as possible and to ensure the reliability of the data. Forms 
were distributed to data collection points one week prior to the beginning of the study and 
collected after the end of the study period. The gathered data was then converted to 
machine-readable form by the study team using the QPL data inputting software. Ba^fd on 
this evaluation, the program HRD-QPU available from the Human Resources Department 
of the GAO (Government Accounting Office), was purchased. QPL (Questionnaire 
Programming Language) has several inter-related components which assist in the 
development of the questionnaire, display it in a form suitable for interviewers, allow direct 
input of the data into a personal computer as it is being collected by the interviewer (or an 
alternative ^keypunch* mode, for manual inputting of data) and convert the collected data 
into any of several formats for importing into statistical or database programs. SPSS-PC+, 
Version 4.0 was used to generate frequency counts, cross-tabulations, and probability 
measures. SPSS-PC+ provided us with the power and flexibility needed for data analysis 
within a group structure. Further, the QPL software provided an option to convert data 
easily into SPSS format. QPL, SPSS-PC+, and the data sets collected during this study were 
all loaded on one PC to which the committee had access. 

6. Budget 

The human and fiscal resources available for the project were limited. We needed to 
create, duplicate, distribute and collect the survey forms; input and analyze the data; and 
share the results with the Library, all within a budget of $1000. The human resources 



included a portion of the time of the eight members of the Measurement and Evaluation 
Subcommittee, the assistance of Prof. James Kluegel, Head of the Sociology Department, 
the UIUC Social Sciences Quantitative Laboratory consultants, and the cooperation of the 
entire Library staff. 

B. Data Collection Instruments 

The Subcommittee examined several methods of collecting data* The option of using 
transaction log data to further enhance data collection was also examined. However, it was 
decided that transaction logs did not capture information about how users identify and 
physically locate periodical articles. It was determined that user surveys distributed to 
patrons known to be seeking journal literature would target users at the appropriate time 
in the research process. Further, the Subcommittee decided to design forms that would 
enable Library staff to keep a running daily tally of activities related to the provision of 
periodical information. 

Questionnaires and other data collection forms were created on a word processor using 
the QPL language and were imported into QPL for data collection and conversion. QPL 
allows the creation of questions with multiple choice answers, "number" answers, short 
phrases, cr dates. A sample page from a survey instrument as constructed with QPL is 
included in Appendix B. When questionnaires were modified, based on the pretest, it was 
straightforward to update the QPL source files to reflect these changes. A sample packet 
of all of the data collection forms and the accompanying staff instructions and information 
is included in Appendix A. 

In the "Reference/Service Point Tally Sheet", we identified three major patron assistance 
functions: finding serials in the UIUC collection, assisting patrons in their use of periodical 
article finding tools, and interpreting the information from periodical article finding tools. 
The first function, finding serials within the Library's collection, was a direct measure of staff 
involvement in the location of serials within the Library and an indirect measure of actual 
use of serials by the Library's users. Since periodical use was difficult to measure, we 
approached it from more than one direction. The second function, assisting users in 
discovering periodical articles, measured staff involvement in using current periodical article 
finding tools. We believed it would provide some insight into how "user friendly" these tools 
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are. It would also reveal current patterns of use among the several approaches currently 
available to our Library's patrons. The third function measured by this survey highlighted 
the staff wor'doad in interpreting citation information that the patron may have found 
(unaided by Library staff) in these tools. 

The "UIUC library Survey" was administered to the library's patrons at 36 sites within 
our system. This survey reported which tools the patron used to discover the existence of 
the first article being sought and the perceived ease of use of these tools. library staff 
members distributed the survey to patrons who either asked a question about a particular 
journal or periodical citation, or patrons who exhibited behavior which clearly indicated they 
were in the process of using periodical sources. 

The form entitled "Re-Shelving, Machine Servicing and Mutilation Statistics' 1 enabled us 
to gather data on all the ways that the Library staff expend time and energy dealing with the 
physical workload of providing access to periodical literature. The merging of these multiple 
topics into one form, in spite of some lack of common ground, resulted in a focus for staff 
on the nuts and bolts aspects of periodical service, as well as on the wear-and-tear to the 
machines which facilitate periodical access in some way. Again, the aim of the survey was 
to identify all the ways in which providing access has an effect on the Library's staff. 

Three methods were used to measure interlibrary borrowing and lending activity, in an 
effort to determine statewide use of our library's periodical collection which was instigated 
by the IO+ system. 

1. ELL Patron Survey. Patrons requesting articles through ILL Borrowing services were 
asked to identify which tool they used to find the citation they were currently seeking. 

2. LTLS Data Form. Interlibrary borrowing requests were examined to determine if the 
unavailability of material owned by UIUC was a factor in the use of inter-library 
borrowing services, or if the material was not owned at UIUC. 

3. IRRC Data Form. Interlibrary lending requests for articles from the UIUC 
collections were examined to determine the format of citation source and the 
material's availability at UIUC. 

C. Data Inputting 

At the end of the survey period, we divided into teams for inputting data. This was the 



most labor-intensive phase of the study. Once the data was in electronic format, extensive 
error-checking was performed before the statistical analysis began. 
D. S tatistical Measures Used 

A combination of statistical measures were used in the analysis of data. Descriptive 
statistics were used throughout to determine the frequency with which activities occurred. 
In addition, one measure of correlation was used, and a two-tailed measure of probability 
was used. The chi-square statistic was used to determine the correlation between dependent 
and independent variables in the same data set within the same year. Typically, a chi- 
square statistic of .4 or less is considered the cut-off point for determining significance. A 
high correlation indicates that a library's average for a particular variable in one year can 
be predicted from the other year. The two-tailed t-test was employed to measure the 
probabiltiy that the occurrence of the means of the same variables for two separate years 
(i.e., 1991, 1992) differing from zero was the result of chance. For example, a probability 
figure of .10 suggests that there is a 10% likelihood that the results are due to chance. 
Typically, .10 represents the upper threshold for considering significance. In the SPS 
program a significance level of less than .00005 is printed as .00000, The very low 
probability suggests that it is highly unlikely that the two variables in the chi-square test are 
independent in the population. 
IV. Results 
A. Equipment and Reshelving Survey 

Among the factors that the survey was intended to measure was increased wear and tear 
on periodical volumes and library equipment as well as any increases in staff time spent in 
relation to volumes and equipment. In fact, as Table 1 shows, there was a 13.95% overall 
increase in this kind of activity from 1991-1992. Although this was not found to be 
statistically significant, there were modest increases in the number of periodical volumes 
shelved (13.94%) and in re-stocking of workstation printers (21.74%). More substantial 
increases were found in PC breakdowns (31.19%), patron assistance (32.33%), and 
workstation printer breakdowns (51.61%). Very large increases were found in reader-printer 
breakdowns (106.52%) and in reader-printer re-stocking (353.85%). Although the increase 
in compact shelving breakdowns was over 30%, in absolute numbers the incidence was very 
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small. 

Decreases occurred in photocopier breakdowns (1547%) and in reported theft and 
mutilation (59.42%). The number of times photocopiers were re-stocked remained 
substantially the same over the period. 
Analysis 

At the outset of the investigation, it was hypothesized that no changes in reshelving of 
bound volumes and unbound issues, no changes in machine servicing, and no changes in 
mutilation would be brought about by the addition of article databases to IO+. In fact, the 
study reveals changes in all of these activities; however, the increase in printer breakdowns 
was the only statistically significant change from 1991 to 1992. This appears to be a logical 
result of increased article database searching. If more IO+ and CD-ROM searches are 
being done, more intensive use of the printers may result in increased printer problems. 

The large increase in stocking and problems with microfilm/microfiche reader-printers 
(which was not statistically significant) was unexpected, but it was not the result of adding 
journal article databases to 10. It can be explained by the fact that the bulk of the 
increased activity was reported in two units, both of wliich had acquired additional 
reader-printers between the two survey periods. Further, one of the units had lowered the 
price of reader-printer copies in April 1991. 

The study revealed that, in absolute numbers, there was a large amount of library staff 
time invested in both survey periods in handling periodicals, in tending the machines used 
to search for them, read them, and copy them, and in assisting patrons. Detailed statistics 
are in Table 1. Adding article databases to IO+ had the effect of increasing staff workload 
related to periodical use, but an undetermined effect on library materials. Therefore, three 
hypotheses were supported: Hypothesis 1-Expanded access would have no significant effect 
on the number of bound serial volumes and unbound issues reshelved; Hypothesis 2- 
Expanded access would not significantly increase the amount of photocopy machine and 
computer printer servicing; and Hypothesis 3--Expanded access woulc not significantly 
increase the amount of serial mutilation. 

B. Reference and Service Point Survey 

In the Reference/Service Point Tally Sheet, the investigators identified three areas where 
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library staff rendered assistance to patrons: finding serials in the UIUC Collection, helping 
patrons use bibliographic tools to locate periodical articles, and interpreting information 
from these tools* 

The first function, finding periodicals within the Library's collections, was a direct 
measure of staff involvement in the location of periodicals within the Library, and a partial 
measure of the actual use of periodicals. In the survey, two types of questions were tallied 
to measure this function, questions about the specific periodical holdings and questions about 
the location of periodicals within our large and decentralized library, comprised of 38 
departmental libraries. 

In Table 2, the results for both the 1991 and 1992 tallies of these measures are shown. 
From 1991 to 1992 there was a gross decrease in the number of questions asked. The 
investigators applied a two-tailed probability test to measure whether the difference of the 
two measures were actually statistically significant. Our findings indicated that for questions 
about location there was a 17.6% probability that the change was due to chance, and for 
questions about holdings, only a 10.1% probability that the change was due to chance. 
Typically a threshhold of 10% (.10 in the table) represents the upper threshhold for 
considering significance. 

The second function, assisting users in the use of bibliographic tools to locate periodical 
articles, measured staff involvement in this area. We believed it would provide some 
indication of how easy or difficuH it was for this user population to use these tools. It would 
also indicate current patterns of use among the several approaches currently available to our 
patrons. In the survey, five types of questions were used to measure this function. Tallies 
were recorded for any question relating to the use of IO+, our online catalog, the use of 
CD-ROM, printed indexes, bibliographies or other lists, or for computer searches done by 
library staff. 

The measures for this second function were indicated in the variable name by those 
beginning with the letter U (for "using") in Table 2. For all measures but the one 
concerning using IO+, the results were not significant. The t value ranged from .372 to .814, 
indicating that there was a fairly high likelihood that the results were due to chance, from 
37.2% to 81.4%. As far as using IO+ was concerned, there was no change reported because 

12 



ERIC 



IS 



these indexes were not available in 1991, and it was the effect of this introduction that we 
attempted to measure. 

The third function, interpreting information from these tools, measured the staff 
workload in interpreting citation information, which we felt would be an indication of the 
level of bibliographic knowledge of our patrons. In the survey, seven types of questions were 
tallied to measure this function. These were questions relating to interpreting information 
from 10 +, CD-ROM, printed indexes, word of mouth, bibliographies, computer searches 
done by Library staff, and other sources not specifically mentioned here. 

In Table 2, the measures for this function are indicated in the variable column by 
variable names starting with I (for ft interpreting ,f ). Again, in spite of the gross differences 
between 1991 and 1992, the likelihood that these differences between the two years occurred 
by chance ranged from 19.7% for questions relating to interpreting searches done by a 
library staff member to 78.1% for questions relating to interpreting information from a 
printed index. 
Analysis 

The only statistically significant change occurring with less than 10% likelihood that the 
differences were due to chance was in the question dealing with holdings information. The 
apparent change between 1991 and 1992 suggests that the introduction of IO+, which 
includes the Library location and call number for journals in each citationn, had a tangible 
effect on the user population. The result of this effect was to reduce the number of 
questions asked about holdings, thereby decreasing staff time devoted to this activity. 
Hypothesis 6 was supported-expanded access would not increase the number of reference 
questions asked about serials. Hypothesis 4 was not supported-Expanded access would have 
no significant effect on the types of citation sources chosen by patrons to identify journal 
articles. Since all the remaining differences between the two years' measurements had a 
greater than 10% likelihood that the difference was due to chance, the investigators 
interpreted this to mean that no significant change occurred. Hence, no further 
interpretation of these data seemed warranted. 

C. Interlibrarv Borrowing and Lending Surveys 

Because of the significant emphasis placed on resource sharing in Illinois, especially 
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at UIUC, a section of this research project was devoted to obtaining measures of the effect 
of expanded electronic access to journal article information on interUbrary loan photocopy 
requests* We examined both the effect of expanded access on our own patrons' requests 
for materials outside the UIUC collection, as well as its effect on the volume and type of 
requests from patrons at other libraries for materials from the UIUC Library collections. 

1. Interlibraiv Borrowing Patron Survey (ILLSURV^ 
Interlibrary loan requests made by. UIUC patrons were examined to test Hypotheses 3, 
4, and 5. The ILL Patron Survey treated Hypotheses 4 and 5 by eliciting patron status as 
well as how patrons identified the requested articles, as shown in Table 3. 

There were 703 valid instances of patrons requesting articles through interlibrary loan 
services in 1991, and 524 valid instances in 1992. These surveys revealed that graduate 
students made up the majority of patrons in need of material not available at UIUC (1991- 
62%; 1992-56.3%). In 1992, faculty (17%), undergraduates (13%), and staff (11%) also 
used the ILL service, but less frequently. 

Table 3 reveals that there was a significant change in the types of resources patrons 
used to identify journal articles between 1991 and 1992. The 1992 survey also attempted to 
elicit information about which IO+ database was used to identify the journal article which 
was being sought; unfortunately, this data was not useful, as 93% of the patrons queried did 
not indicate which IO+ database they were using. However, it was possible to obtain 
meaningful information about the types of resources used to identify journal articles by 
collapsing many categories into three: 1) ELECTRONIC-electronic resources; 
2) LIBRARY, PRINTED-printed indexing and abstracting services; and 3) PATRON- 
SUPPLIED-a citation supplied from a bibliography or through word of mouth. Overall, 
there was a significant shift from the use of LIBRARY, PRINTED resources to 
ELECTRONIC resources to identify journal article citations. In 1991, over 35% of the ILL 
patrons used LIBRARY, PRINTED resources and 33% used ELECTRONIC resources. 
In 1992, this usage had shifted to 20% LIBRARY, PRINTED resources and 43% 
ELECTRONIC resources. PATRON-SUPPLIED resources increased slightly from 32% to 
37%. These changes were found to be significant at the .00000 level. 

Analysis 
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Hypothesis 5~expanded access would not increase the amount of interlibrary loan 
requests for photocopies of periodical articles, was supported. A change in the methods 
used by patrons to obtain citations for these requests was apparent. Therefore, Hypothesis 
4, which stated that expanded access would not change the types of citation sources used by 
patrons to identify journal articles, was not supported. This analysis suggested a shift away 
from patrons using print indices toward the use of electronic resources such as ELLINET 
Online +, to locate citations for journal article . 

2. Interlibrary Borrowing (Data Form (LTLS form) 

In this section of the study, Library staff members took the information from the 
patrons' periodical article request forms and searched the Online Catalog and the shelves 
to verify the availability of the requested items. The categories were combined into larger 
units to allow a meaningful aggregate view of the data, as shown in Table 4, The first line, 
Owned, Available, showed the articles that were actually owned in the collection, for which 
status was accurately reflected in the Online Catalog, but that were not found by the patron. 
Comparing the data from 1991 to 1992 revealed that there was little change in the actual 
availability of periodical articles in the UIUC collections which had mistakenly been 
requested by patrons. This error rate was consistent with historical information collected by 
this unit. 

The second line in the table, Owned, NOS (not on shelf), reflected those items which 
appeared as an owned title in the Online Catalog, but which were not on the shelf because 
they were in circulation, lost, stolen, or unavailable due to mutilation. There was a reduction 
in this category from 11% in 1991 to 5% in 1992, but the absolute numbers for these 
percentages were quite small, and thus precluded deeper interpretation. The third line in 
Table 4, Not Owned, included those items for which the Online Catalog accurately reflected 
that the title or the item containing the needed article was not in the collection. Overall, 
there was no evidence that availability of materials decreased significantly from one year to 
the next. 

3. Interlibrary Lending (Illinois Research and Reference Center") Data Analysis 
IRRC data included requests from non-UIUC individuals and libraries that seek 
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materials from the UIUCs collections. The IREC Data Form resulted in 3,425 valid cases 
for 1991 and 1,959 for 1992 (Table 6). In both years, the majority of requests (75.3%) in 
1991, and 67.9% in 1992 were Owned by the Library. Reported Instances of a title or item 
not being owned by the Library (Not Owned) were 541 (16,2%) in 1991, and 345 (18.2%) 
in 1992. Items which the Online Catalog reported that the Library owned, but were not 
found on the shelves due to being in circulation, missing, or mutilated, (Owned, NOS), 
accounted for 8.5% of the total in 1991, and 13.9% of the total for 1992, Although staff cuts 
may have curtailed the process of checking the shelves twice for items in 1992, it is doubtful 
that this was the sole factor that could account for the large increase in items not on the 
shelves from 1991 to 1992. The percentage of requests that matched the Online Catalog 
record (Owned- Avail.) declined from 73.3% in 1991 to 65.9% in 1992. Thus, Hypothesis 
3 was not supported-Expanded. 

In both 1991 and 1992, the majority of requests had no verification indicated (Table 6). 
Table 6 also revealed an increase in the number of requests which were verified using an 
electronic resource for 1992. 

Although electronic resources comprised a larger portion of verification sources used 
compared to traditional sources, the end result was not a higher proportion of verified 
requests, but rather a slight decrease in verified requests. This could suggest that patrons 
see less need to record their source when the ease of use of that source or the frequency 
with which they use that source increases. 

Although the overall numbers were small, the increase in verification using Electronic 
resources from 2.6% in 1991 to 7.7% in 1992 was significant, and thus does not support 
Hypothesis 4--Expanded access would have no significant effect on the types of citation 
sources chosen by patrons to identify journal articles. 
Analysis 

A cursory comparison of the number of requests for photocopies of journal articles 
suggests that Hypothesis 5 was supported: Expanded access did not significantly increase 
the amount of interlibrary loan requests for photocopies of journal articles. 
However, it is important to note that the IRRC underwent both budget and staffing cuts, 
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significant changes which could be construed to challenge two of the assumptions of our 
study. Although the extent of the effect of these cuts on the results of this study cannot be 
determined, the reader should be aware that these external factors may have affected the 
results of this portion of the study. Hypothesis 3 was supported. Hypothesis 4 was not 
supported by the data collected, indicating that there was a significant shift in sources 
patrons used to find journal article citations. 
C. Patron Survey 

The final area of analysis was the survey conducted among patrons who were looking 
for periodical articles in the UIUC Library. Our objective was to identify the major 
elements of user behavior in their identification and use of periodical articles. The survey 
included questions about how often the Online Catalog was used, how difficult or easy it was 
to find periodical information in the Online Catalog, and how much time was required to 
compile a list of citations. 

After preliminary analysis, the methods for finding articles were clustered into 
ELECTRONIC, LIBRARY, PRINTED, and PATRON-SUPPLIED. In Table 7, the 
correlation between the year (YEAR) and the method used for finding journal articles 
(FINDCLUS) showed a substantial increase from 1991 to 1992. In 1991, only 25% of the 
patrons identified an electronic resource as the source of a citation. However, by 1992, the 
category of electronic resources was cited by nearly 52% of the patrons surveyed as the type 
of resource used to find journal articles. This change represented a significant shift in 
patron periodical seeking behavior. 

We also investigated whether some sectors of our patron population made this shift 
more readily than others. Hypothesis 9 examined the frequency of use of the Online 
Catalog as it related to using electronic resources. Tables 8 and 9 revealed that patrons who 
indicated they used the online catalog at different frequencies exhibited different behavior 
in finding out about periodical articles. We divided online catalog users into three catagories 
that are represented in Tables 8 and 9 by these upper-case terms: FREQUENT, 
MODERATE, and RARELY (the terms themselves referring to the frequency of online 
catalog use indicated by the patron on the survey). The Subcommittee felt that these data 
would be of particular interest to the Online Catalog Advisory Committee as well as 
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individual staff members who work closely with the Online Catalog. 

Some especially interesting results results are discussed here. In 1991, even the frequent 
users of the Online Catalog used electronic resources to find journal citations infrequently, 
approximately 20% of the time, while patron-supplied was the dominant resource with nearly 
60%. Among moderate users in 1991, resource use was split evenly among the three 
sources: 33% electronic, 36% library, printed, and 32% patron-supplied. Among the group 
that uses the Online Catalog rarely, the resources used were divided between library, printed 
and patron-supplied at 40% and 38% respectively, with 22% using electronic resources. In 
1992, the patterns revealed a substantially different picture. Among frequent users of the 
Online Catalog, 50% used electronic resources, with patron-supplied falling to 35% and 
library, printed plummeting to 15%. Similarly, moderate online catalog users shifted 
dramatically from 32.9% use of eiw mic resources in 1991 to 58.7% use of the same 
resources in 1992. Conversely, between 1991 and 1992 patrons who used the catalog rarely 
shifted away from using library, printed resources, some using electronic, but more relying 
on patron-supplied. These figures are significant at the .00000 level. Thus the date show 
that Hypothesis 9 was not supported. Conversely, the findings indicate that the frequency 
with which the patron uses the online catalog is strongly predictive of the use of electronic 
resources for finding journal articles. 

Another method chosen for examining the changes in user behavior was to analyze the 
data by subject area. All the patron data surveys were identified by departmental library 
location. The survey data were grouped according to the existing councils which represent 
libaries clustered into broad subject areas, and Central Public Services, which is comprised 
of library units that serve the entire campus. Tables 10 and 11, (COUNCIL by FINDCLUS 
controlling for YEAR), enabled us to examine journal article seeking behavior of users in 
libraries clustered around the following broad subject and functional areas: Humanities 
(HUM), life Sciences (LIF), Physical Sciences (PHY), Social Sciences (SOC), and Central 
Public Services (CPS). The second row of figures in each cell represents the percentage of 
that council's patrons using that clustered resource. In 1991, only 6.7% of the Humanities 
patrons used an electronic resource to identify periodical articles. This contrasts with the 
18% of the life Sciences patrons, 16% of the Physical Sciences patrons and the 37% of 
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Social Science patrons who reported using electronic resources in 1991. The 1992 < amn 
for electronic resource use reveals the following percentages: 34% of Humanities, 67% of 
Life Sciences, 46% of Physical Sciences and 56% of Social Science patrons were using 
electronic resources to find journal articles. Since the Central Public Services (CPS) row 
represents patrons from all subject areas in unknown proportions, this row is less useful for 
identifying subject differences. Patrons of libraries in the Central Public Services cluster, 
which had a mid-range percentage in 1991 of 28% is again in the mid-range of the electronic 
revolution with 51% of the patrons having used electronic resources to discover their journal 
articles. 

Analysis 

It is difficult to determine whether users' shift from the use of traditional print library 
resources or from self-supplied sources to the use of electronic resources in finding journal 
articles is an improvement in their success in finding journal articles, or simply a change in 
the methods which they employ. We suggest it is both: Some patrons who literally could 
not find information on their topic in traditional sources, because of the narrowness or 
newness of the topic, were able to find the same information usiiig the keyword searching 
options which the ILLINET Online Plus search user interface software now offers. For 
other users who are not well-informed about the nature of searching for journal articles by 
using print sources, the use of electronic journal citation databases may not be advantageous 
without some prior instruction. 

The information in Tables 10 and 11 reflect the diversity of approaches to finding journal 
articles of patrons using the libraries across broad groupings of library units. The changes 
from 1991 to 1992 in the use of electronic reference sources may be reflective of the array 
of sources, or of differing patterns of research, or a combination. Since the link between the 
Library council of the survey and subject discipline is physical, direct conclusions about the 
role of electronic reference sources or about the nature of journal use in different disciplines 
are not possible. 
V. Conclusions and Summary 

A* External factors and Limitations 

There were several limitations on this study. Perhaps the most important one was the 
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sue and representativeness of the sample taken. Although the investigators assumed that 
thit was a representative sample, the nature of the data collection rendered it impossible to 
verify this. No legitimate sampling method was employed. Because of the large number of 
data collection sites and number of personnel involved in data collection, there was no 
scientific assurance that data was collected without bias, or that a representative sample of 
journal article users was chosen. By choosing similar two-week periods one year apart, 
however, the investigators intended to introduce some validity in the measurements taken. 
Further, by training the data collectors, reliability of the resulting data was strengthened. 
Although a more rigorous experimental design would have increased the reliability of data 
and causality among variables, it would have introduced changes to service provision that 
would not have been representative of the actual Library environment. We believed that 
broad participation in a field study with practical applications was more fitting to our 
research than attempting to strictly control this particular experimental environment. 

Budget and staff cuts were two externa! factors which affected the outcome of the 
second year of the Interlibrary lending and borrowing surveys and data collection. The 
University Library's interlibrary lending office, the Dlinois Research and Reference Center 
(IRRC), experienced pronounced changes in funding support in the fiscal year of 1992 that 
affected staffing levels and several policies and procedures on which its processing practices 
are dependent. Therefore, the changes described do not support Assumption 3-that ease 
of physically borrowing an item would not change, and Assumption 7-that there would be 
no change in interlibrary loan procedures that would affect the number of requests made. 
Although "the ease of physically borrowing an item either on-site or through interlibrary loan 
did not change" for UIUC patrons, it did change for patrons (and libraries) requesting 
journal article photocopies from the UIUC Library's collections. The fact that the IRRCs 
ability to support any potential increase in requests for ELL photocopies of journal articles 
was diminished undoubtedly affected the outcome of this portion of the study. The reader 
should be aware of these factors, and the limitations on the validity of the IRRC data and 
its resulting analysis. 

The Subcommittee realizes the very real limits of the study. These limits are due to the 
inability to control or measure the underlying assumptions; thus, we were unable to 

20 

*?6 



determine any role potential changes in these assumptions might play in the results. In some 
cases, we measured activities that occurred too infrequently tc be meaningful in the two- 
week time samples of the study, like compact shelving jams or reports of journal thesft or 
multilation. 

The strength of this research design came from its reliance on simple data gathering 
techniques which are second nature to virtually all library public service staff. It departs 
from the norm by not relying solely on patrons' reported uses of periodical literature. This 
study enabled us to map the intricate organizational network that supports periodical usage 
at a given access level. The study sought to measure the current levels of journal access, 
interlibrary loan, reference assistance, demands on equipment and staff, and users' journal 
article seeking behavior. These data are revealing because they highlight current patterns 
of use and staff involvement in the provision of periodical information. Informal feedback 
from Library staff confirmed that the data collection instruments elicited measurable data to 
describe the instances of service they provide most frequently, but which have not heretofore 
been quantified. 

User survey participants were self-selected due to the fact that they only received the 
survey if they were looking for a specific periodical citation. However, each library reported 
a high rate of return on the surveys which they distributed. A high level of participation and 
attention to detail was evident in questions and feedback from both staff and patrons 
throughout the data collection period. Users appeared eager to have input concerning their 
information-seeking habits. 

B. Su pport for hypotheses 
This section presents a summary of the hypotheses, and an indication of how each was 
supported by the data analysis: 

HI. Expanded access would have no significant effect on the amount of bound 
serial volumes and unbound issues reshelved. 
Supported (Equipment and reshelving). 

H2. Expanded access would not significantly increase the amount of photocopy 

machine and computer printer servicing. 

Supported (Equipment and reshelving). 
H3. Expanded access would not significantly increase the amount of serial 
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mutilation. 

Supported. (Equipment and reshelving). 
H4. Expanded access would have no significant effect on the types of citation 

sources chosen by patrons to identify journal articles. 

Not supported (ILL, Patron). 
H5. Expanded access would not significantly increase the amount of interlibrary 

loan requests for photocopies of journal articles. 

Supported (ILL, IRRC). 

H6. Expanded access would not significantly increase the number of reference 
questions asked about serials. 
Supported (Reference and Service point). 

H7. Expanded access would not cause a significant change in the type of reference 
questions asked about serials. 
Not supported (Reference and Service point). 

H8. A user's status in the University community is not a predictor of journal article 
seeking behavior. 

Not supported (ILL, Patron surveys). 

H9. The frequency with which a patron uses the online catalog is not a predictor 
of use of IO+. 

Not supported (Patron survey). 

H10 Expanded access would not cause a significant change in the journal article 
seeking behavior among users in libraries clustered into broad subject 
disciplines (Arts and Humanities, Life Sciences, Physical Sciences, Social 
Sciences, General population (Reference, Documents, Newspaper, 
Undergraduate, University High School, Circulation and Bookstacks). 
Not supported (Patron survey). 



In brief, the results of this research suggest that enhanced electronic access to journal 
article information has resulted in a slight increase in the use of periodicals, but this increase 
was not found to be statistically significant. Conversely, enhanced electronic access to 
journal article citations has caused a significant change in the choice of resources users 
employ to find journal articles. 

At the outset of this study there was a perception among library staff that increased 
electronic access to periodical literature would result in a significant increase in the attendant 
support activities-stocking and repairing printers, photocopiers, reader/printers, and re- 
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shelving of bound volumes and unbound issues of journals. Table 1 clearly reflects an 
increase in these activities from 1991 to 1992; however, these are not statistically significant 
increases. These results should not be interpreted as contradictory. Rather, the reason for 
the uninteresting results is the fact that the time sample is too limited. The size of the 
population is appropriate, since it represents the universe of libraries on this campus, but 
the events covered in the survey (asking questions about using CD-ROM resources, journal 
holdings, compact shelving jams) are rare occurrences in the continuum of total library 
activities. Thus, if we wish to re-test the hypotheses represented by these data, it would be 
necessary to expand the data collection over a considerably larger sample of time than that 
chosen for this study. 

If we think in terms of how users search for journal articles, enhanced electronic 
access to this information has resulted in an increase in the use of ELLINET Online Plus to 
find journal article citations; however, these data suggested that it did not necessarily follow 
that users retrieved and photocopied more journal articles in 1992 than they did in 1991, 
prior to the introduction of the IO+ journal citation databases. 

There is some evidence that patrons wished to obtain articles that the Library 
increasingly does not own (through failure to own the needed volumes of a particular title, 
or through failure to hold the title). It is recommended that the Library undertake further 
exploration of this situation. 
VI. Recommendations 

The 1991 data alone indicates that before the introduction of enhanced electronic 
access to journal citations through ILLINET Online Plus, the Library carried on a brisk 
trade in providing information and support services related to patrons' use of periodicals. 
Our comparison of the 1991 data with the 1992 data revealed that with enhanced electronic 
access to journal article information, UIUC Library users have readily embraced change at 
a rate that may have already surpassed the Library's ability to provide the necessary user 
services and assistance. 

Based on the results of this study, the subcommittee makes several recommendations 
for Library consideration. First, users' willingness to migrate to electronic resources suggests 
that the Library ought to further develop our existing service expertise by integrating a cohesive 
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program of training in electronic information resources for staff at appropriate levels. Well- 
developed and continued training programs can maintain staff expertise at the information 
resource-finding forefront, enabling them to share expertise, and to apply that knowledge 
by developing instructional programs and materials for diverse user populations. 

Second, there is some indication that patron preference for using electronic resources 
to find journal citations will increase significantly. Therefore, we recommend that the Library 
continue to support the addition of more journal citation databases to ILUNET Online Plus, 
or through networked access to CD-ROM or other available and appropriate databases. 
Increased electronic access will necessitate increased physical accessibility to IO+. In order 
to accommodate this need, we recommend that a rational plan for adding public access 
terminals be developed and implemented throughout the D*>rary. 

Due to the fact that the time sample was too small, the crucial information about the 
effect of enhanced electronic access on staff activities which support users* journal article 
seeking was inconclusive. Current decision-making about technology and where to put it 
could be augmented if the Library were to more closely monitor the activities related to 
journal article seeking. We recommend that the Library devise and implement ongoing 
methods to collect, examine, and analyze data that reflect the effect of electronic access to 
journal article information on staff, collections, and users. 

Finally, we have identified two areas which, although they are related to this study, 
did not fall directly within the scope of our examination, but which deserve further 
investigation. Further investigation of theft and mutilation of periodicals is recommended. 
Finally, we recommend further study of the factors that can cause an increase in 
unavailability of journal articles (e.g., journal cancellations; availabilty of staff to retrieve this 
information). 
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APPENDIX A—DATA COLLECTION INSTRUMENTS 1991 & 1992 



UIUC LIBRARY SURVEY - INSTRUCTIONS FOR IMPLEMENTATION 



Library Unit; This line is to be filled in by the sendee point, in advance of giving it to the 
patron. If a library has more than one service point, they should be 
distinguished. For example, Reference Library • North Desk. 

Date: This line is to be filled in by the service point, in advance of giving it to the 
patron. 

General Administration Instructions: 

Telephone patrons are excluded from the scope of this patron survey. 

It is important that each patron who is seeking to locate an article within the UIUC 
Library during the survey period is asked to fill out a survey form. It is also important that 
the patron fill out only 1 form per visit. Each f rip to the Library is a new unit of 
measurement. Ask the patron if he/she has competed a survey form on THIS visit to the 
Library. 

The survey is designed to reach the patrons who have already used some source of 
information to identify one (or more) particular articles of interest to them at the time. For 
example, the survey form is to be given to those patrons who are asking for 'This article by 
J. Jones in the journal Time is Money ." It is no; *or those patrons who say "I need to find 
three articles on my topic." 

The survey is to be handed to the patron on the clipboard with a pen/pencil. 

It will improve the survey return rate if you ask the patron to return the form to 
yourself. It will help if you are able to make the importance of the form clear to the patron. 

Thank the patron for agreeing to participate- Please indicate to the patron that it is 
a VERY brief survey and should take no more than a minute or two to complete. 

When the survey is returned, please scan it for completeness. If it seems reasonable, 
ask the patron to complete if there are blanks. 

Each completed survey form should be put in a specified place at the service point. 
The completed forms should be collected in each unit each day of the two week survey 
period and sent together to Beth Sandore, 220 Library, on Monday, February 17, 1992. 
Questions about correctly interpreting the guidelines should be directed to any member of 
the Subcommittee: Sandore, Block, Burger, Chaplan, Kluegel, Newsome, L. Romero, or 
Stenstrom. 

Thank you for your assistance in administering this survey. 
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SERVICE POINT: 



REFERENCE/SERVICE POINT TALLY SHEET 

DATE:. 

(MAKE A HASH MARK FOR EACH TIME THIS OCCURS.) 



A QUESTIONS ABOUT SERIALS: TOTALS: 

HOLDINGS 

LOCATIONS 



B. HOW TO USE: 

Online Catalog (IO+) References to Articles 

e.g., IBIS, ERIC, CARL 

CD-ROM, e.g., WILSONDISC 

PRINTED INDEX, e.g., READERS' GUIDE 

BIBLIOGRAPHY OR OTHER LIST 



COMPUTER SEARCH DONE BY LIB. STF. 



C. INTERPRETING INFORMATION FROM: 

Online Catalog (IO+) References to Articles 

e.g., IBIS, ERIC, CARL 

CD-ROM, e.g., WILSONDISC 

PRINTED INDEX, e.g., READERS' GUIDE 

WORD OF MOUTH 



BIBLIOGRAPHY OR OTHER LIST 



COMPUTER SEARCH DONE BY LIB. STF. 



OTHER (Please specify) 



(All forms should be returned together to B. Sandore, 220 Library, on Feb. 17, 1992.) 

Please see reverse for instructions. 

(refstf.pst 2/92) 
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UIUC LIBRARY SURVEY 
LIBRARY UNIT: ' DATE: 



*** PLE ASE PUT A CHECK MARK BY ONLY ONE ANSWER FOR EACH QUESTION *** THANK YOU *** 

1. ARE YOU TRYING TO LOCATE ONE (OR MORE) SPECIFIC ARTICLE(S) THAT YOU HAVE 
IDENTIFIED? 

If NO, please return to the library staff member. If YES, continue. 

2. HOW DID YOU FIND OUT ABOUT THE FIRST ARTICLE YOU ARE LOOKING FOR NOW? 
CD-ROM index, e.g., WILSONDISC, InfoTrac, SilverPlatter 

Printed index, e.g., Readers* Guide 

Word of mouth, e.g., instructor, colleague, friend 

Bibliography, course reading, or other list, e.g., endnotes 

__ Computer search done by library staff member 

Other (please specify) 

3. HOW MANY ARTICLES ARE YOU LOOKING FOR NOW? 
1 2-5 6-20 20+ 



4. HOW MANY MINUTES DID IT TAKE TO PRODUCE THE LIST OF ARTICLES) YOU ARE 
LOOKING FOR NOW? 

Less than 5 5-15 15-30 30-60 60-120 120+ 



5. HOW HARD WAS IT TO PRODUCE THE LIST OF ARTICLE(S) YOU ARE LOOKING FOR NOW? 
Very Hard Hard Not too Hard Fairly Easy Easy 

6. HOW OFTEN DO YOU USE THE ONLINE CATALOG, ON AVERAGE? 
Every day 1-3 times per semester 

1-6 times per week 1 time per year 

1-3 times per month never 

7. WHAT IS YOUR STATUS AT THE UNIVERSITY OF ILLINOIS? 

Undergraduate Graduate Faculty Staff Other 

PLEASE RETURN THIS FORM TO A LIBRARY STAFF MEMBER 

(patron.pre) 1/91 




APPENDIX B — SAMPLE PAGE, QPL SOURCE CODE 



**** ********************************* 

** QPL SOURCE FILE ** 

** ** 

** KATHLEEN KLUEGEL - EDITION 1.0 ** 

** ** 

** Patron Survey - 1992 ** 

**** ******************************* ** 

.CARD - 60 

.TITLE = "Patron Survey - 1992" 



.QUESTION - DATE 
.TYPE - XDATE 
What is today's date? 
.ANSWER 

.NEXT = IDPATRON 



* Computer date of data input 



♦Computer supplies automatically 



.ESCAPE - FINISH 



* Press Escape key to end data entry 



.QUESTION - IDPATRON 

.TYPE = NUMBER 

Type in record ID number. 

.ANSWER - 4 

.NEXT 



* Record ID Number - Patron Survey 

* Sequential ID number 

* Up to a 4 Digit number 



.QUESTION = SURVDATE 

.TYPE = NUMBER = "##-##-##" 

What date was survey completed. 

.ANSWER 

.NEXT 



* Date Survey Completed 

* Format is MODAYR 
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.QUESTION * LIBRARY 
.TYPE - STRING 
Library unit: 
.ANSWER = 3 



* Library Unit 



* Enter 3 letter Library code. Info Desk - INF 

48 
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TABLE 3: ILL SURVEY-CROSSTABS-STATUS OF REQUUTOR BY HOW THE 
CITATION WAS FOUND, 1991 & 1992 



1991 


Count 
Row Pet 
Col Pet 


Undergrad 


Graduate 


Faculty 


Staff 


Other 


Row | 
Total 


ELECTRONIC 


1.00 


11 


163 


28 


23 


6 


231.000 






.048 


.706 


.121 


.100 


.026 


0.331 






.190 


.374 


.280 


.230 


.667 




PRINTED 


2.00 


30 


133 


32 


51 


2 


248.000 


LIBRARY 




.121 


.536 


.129 


.206 


.008 


0355 






.517 


.305 


.320 


.510 


.222 




SELF- 


3.00 


17 


140 


40 


26 


1 


224.000 


SUPPLIED 




.076 


.625 


.179 


.116 


.004 


0.321 






.293 


.321 


.400 


.260 


.111 






Column 


58.000 


436.000 


100.000 


100.000 


9.000 


703.000 




Total 


0.0S3 


0.620 


0.142 


0.142 


0.013 


1.000 



1992 


Count 
Row Pet 
Col Pet 


Undergrad 


Graduate 


Faculty 


Staff 


Other 


Row 
Total 


ELECTRONIC 


1.00 


29 


139 


21 


35 


2 


226.000 






.128 


.615 


.093 


.155 


.009 


0.431 






.372 


.471 


.231 


.614 


.667 




PRINTED 


2.00 


34 


58 


8 


4 


1 


105.000 


LIBRARY 




.324 


.552 


.076 


.038 


.001 


0.20 






.436 


.197 


.088 


.07 


.333 




SELF- 


3.00 


15 


98 


62 


18 




193.000 


SUPPLIED 




.078 
.192 


.508 
.332 


.321 
.681 


.093 
.316 




0.368 




Column 


78.000 


295.000 


91.000 


57.000 


3.000 


524.000 




Total 


0.149 


0.563 


0.174 


0.109 


0.006 


1.000 



Chi-Square Value; DF Significance 



Pearson 80.797p7 8 .00000 

Number of missing observations: 130 
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TABLES: 



IRRC DATA FORM-CROSSTABS-STATUS BY YEAR, 1991 & 1992 







1991 


1992 


Row 




K.OW rCl 






Total 




Col Pet 


1.00 


2.00 




ILLINOIS 


1.00 


2083 


1016 


3,099.000 






.672 


.328 


0.576 






.609 


.519 




NON-ILLINOIS 


2.00 


1337 


943 


2,280.000 






.586 


.414 


0.424 






.391 

i 1 


.481 






Column 


3,420.000 


1,959.000 


5379.000 




Total 


0.636 


0.364 


1.000 






Chi-Square 


Value 


DF 


Significance 




Pearson 


41.71126 


r~ 


.00000 





Number of Missing Observations: 5 
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TABLE & 1RRC DATA FORM-CROSSTABS-VERIHCATION SOURCE (FINDCLUS) 
BY YEAR, 1991 & 1992 



YEAR 


Count 
Row Pet 


ELECTRONIC 


PRINTED 
LIBRARY 


NONE OR 
OTHER 


Row 
Total 


„ 


1.00 


88 


685 


2652 


3,425.000 


1991 




.368 


.773 


.623 


0.636 




2.00 


151 


201 


1607 


1,959.000 


1992 




.632 


.227 


.377 


0.364 




Column 


239.000 


886.000 


4,259.000 


5384.000 




Total 


0.044 


0.165 


0.791 


1.000 




Chi-Square 


Value 


DF 


Significance 






Pearson 


149.30293 


2~ 


.00000 
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TABLE 7: PATRON SURVEY, 1991 & 1992-CROSSTABS-YEAR BY SOURCE OF 
j CITATION (FINDCLUS) 





Count 




PRINTED 


SELF- 


Row 


YEAR 


Row Pet 


ELECTRONIC 


LIBRARY 


SUPPLIED 


Total 




i on 


XOJL 


£AJt 






1991 


i 


.256 


.292 


.452 


0.457 




2.00 


437 


142 


263 


842.000 


1992 




.519 


.169 


.312 


0.543 




Column 


618.000 


349.000 


583.000 


1,550.000 




Total 


0.399 


0.225 


0.376 


1.000 



j 

Number of Missing Observations: 4 
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TABLE & PATRON SURVEY, 1991 - CROSSTABS - FREQUENCY OF ONLINE 
CATALOG USE BY SOURCE OF CITATION (FINDCLUS) 





Count 
Row Pet 
Col Pet 


FREQUENT 


MODERATE 


RARELY 


Row 
Total 




1 fin 






XV 








.326 


.586 


.088 


0.259 






.195 


.329 


.216 




PRINTED 


9 00 


58 


116 


30 


204000 


LIBRARY 




.284 


.569 


.147 


0292 






.191 


.360 


.405 




SELF- 


3.00 


186 


100 


28 


314.000 


SUPPLIED 


i 


.592 


.318 


.089 


0.449 






.614 


311 


.378 






Column 
Total 


303.000 
0.434 


322.000 
0.461 


74.000 
0.106 


699.000 
1.000 



DF Significance 
4~ .00000 



Chi-Square Value 



Pearson 63.28075 



TABLE 9: 



PATRON SURVEY, 1992-CROSSTABS-FREQUENCY OF ONLINE 
CATALOG USE BY SOURCE OF CITATION (FINDCLUS) 





Count 








Row 




Row Pet 


FREQUENT 


MODERATE 


RARELY 


Total 




Col Pet 










ELECTRONIC 


1.00 


191 


216 


21 


428.000 






.446 


.505 


.049 


0.519 






.503 


.587 


.276 




PRINTED 


2.00 


58 


59 


22 


139.000 


LIBRARY 




jit 

.417 


ATA 




u.ioy 






,10 J 


•iOU 






SELF- 


3.00 


131 


93 


33 


257.000 


SUPPLIED 




.510 


.362 


.128 


0312 






.345 


.253 


.434 






Column 


380.000 


368.000 


76.000 


824.000 




Total 


0.461 


0.447 


0.092 


1.000 




Chi-Square 


Value 


DF 


Significance 






Pearson 


28.48294 


4~ 


.00001 







ERIC 
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TABLE 10: PATRON SURVEY, 1991-CROSSTABS-COUNCIL BY SOURCE OF CITATION 





Count 




PRINTED 


SELF- 


Row 


COUNCIL 


Row Pet 


ELECTRONIC 


LIBRARY 


SUPPLIED 


Total 




Col Pet 










CENTRAL 


CPS 


100 


102 


150 


352.000 


PUBLIC 




.284 


.290 


.426 


0.497 


SERVICES 




.552 


.493 


.469 






HUM 


3 


17 


25 


45.000 


HUMANITIES 




.067 


.378 


.556 


0.064 






.017 


.082 


.078 




LIFE 


LiF 


12 


21 


33 


66.000 


SCIENCES 




.182 


.318 


.500 


0.093 






.066 


.101 


.103 




PHYSICAL 


PHY 


19 


39 


61 


119.000 


SCIENCES 




.160 


.328 


.513 


0.168 






.105 


.188 


.191 




SOCIAL 


SOC 


47 


28 


51 


126.000 


SCIENCES 




.373 


.222 


.405 


0.178 






.260 


.135 


.159 






Column 


181.000 


207.000 


320.000 


708.000 




Total 


0.256 


0.292 


0.452 


1.000 



Chi-Square Value DF Significance 

Pearson 27.31028 8~~ .00062 
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TABLE 11: PATRON SURVEY, 1992-CROSSTABS-COUNCIL BY SOURCE OFClTATION(JTNDCLUS) 





Count 




PRINTED 


SELF- 


Row 


COUNCIL 


Row Pet 


ELECTRONIC 


LIBRARY 


SUPPLIED 


Total 




Col Pet 










CENTRAL 


CPS 


263 


83 


171 


517.000 


PUBLIC 




.5G9 


.161 


331 


.614 


SERVICES 




.602 


.585 


.650 






HUM 


12 


7 


16 


35.000 


HUMANITIES 




.343 


.200 


.457 


0.042 






.027 


.049 


.061 




LIFE 


LIF 


54 


7 


19 


80.000 


SCIENCES 




.675 


.088 


.238 


.095 






.124 


.049 


.072 




PHYSICAL 


PHY 


48 


17 


38 


103.000 


SCIENCES 




.466 


.165 


.369 


.122 






.110 


.120 


.144 




SOCIAL 


SOC 


60 


28 


19 


107.000 


SCIENCES 




.561 


.262 


.178 


.127 






.137 


.197 


.072 






Column 


437.000 


142.000 


263.000 


842.000 




Total 


0.519 


0.169 


0.312 


1.000 



Chi-Square Value DF Significance 

Pearson 27.51744 8~ .00058 



